Add Android sample (JNI binding + Java wrapper class) #101
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds
examples/clip.android, an Android sample that interfaces with the shared libraries ofclip.cppthrough JNI. The structure of the project is highly inspired fromllama.cpp/examples/llama.android.Summary of changes
examples/clip.androidis an Android project with two modulesappandclip.appmodule contains the sources for a demo application which allows the user to select an image and enter a text description to compare them (uses Kotlin/Compose)clipmodule containsclip/src/main/cpp/clip-androidthat contains the JNI interfaces which allow communication with the underlying functions fromclip.h.clipmodule containsclip/src/main/java/android/clip/cpp/CLIPAndroid.javawhich is wrapper class containingnativemethods.clipmodule also houses an instrumented testclip/src/androidTest/java/android/example/clip/CLIPInstrumentedTest.ktwritten in KotlinREADME.mdis included in theexamples/clip.androiddirectory which describes how to run the sample app and the instrumented testclip.cppis modified to address issue (Possible) Incorrect assignment of image mean/std values inclip_model_load()#99CMakeLists.txtis modified to add-mcpu=nativewhen${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64"only if-DCLIP_NATIVEis enabledclip_image_preprocess_no_resizeinclip.handclip.cppwhich does works exactly likeclip_image_preprocessbut does not perform linear interpolation (no image resizing). Android'sBitmapclass and Python's PIL can handle bilinear interpolation easily, so it would be great to have a method that skips resizing.Note
Though the
clipmodule here in built as an Android library module, it can be built into a 'Java library' module (packaged as a JAR) and used in other Java-only projects as a port toclip.cppIt was fun working on this sample, and I would like @monatis to review it and share feedback.